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FOREWORD 



In a small and wise book published in 1971, Alice M. Rivlin — no stranger to public policies 
for higher education — provides two cautions for the development and use of pertoi-mance 
indicators: "First, single measures of performance should be avoided. . . . Second, performance 
measures should reflect the difficulty of the problem.'" Simplistic, single measures reflect only 
simple, singular things. Sound and sobering advice as we in higher educa^on now scurry to 
comply with a new federal law requiring one "graduation rate" statistic for each of the more than 
3,000 higher education institutions and many smaller postsecondary providers across the nation. 

We could, presumably, observe Rivlin's precautions and disregard the new law, were it not for 
the severe financial consequences of non-compliance., We could disregard the increasingly 
frequent calls for performance and "outcomes" measures in higher education, were it not for the 
considerable political weight of an "education president" and 50 state governors, raising our 
expectations for education in a nearly unanimous voice. We could disregard this challenge if we 
truly lacked the intellectual and technical resources to design and provide meaningful 
performance measures, or if we did not believe in the potential of good analysis and meaningful 
indicators of progress to improve education. Rivlin concludes her small book on a positive note 
with a chapter that explores the meaning anu the demand for greater "accountability" — a 
prophetic call in 1971, a proverbial cry in 1991. "Put more simply, to do better, we must have 
a way of distinguishing better from worse."^ 

Public Law 101-542, The Student Right-to-Know and Campus Security Act of 1990, is all about 
knowing better from worse. Put simply, one section of this law requires regular public disclosure 
of a "graduation rate" by all postsecondary institutions, defined as the proportion of full-time 
entering students who complete a degree or certificate program within 150% of "normal time." 
A second major section requires more detailed reporting of graduation rates by institutions that 
provide financial assistance to students contingent upon panicipation in intercollegiate athletics. 
The third major component requires disclosure and reporting of campus-based crimes and public 
safety policies. The thrust and the potential effects of the law rest upon a few, seemingly simple 
statistics on higher education — one to inform the consumers, one to make sure that education 
is reasonably provided, and one to protect those who are engaged in that education. 

This is the context and the challenge which Peter T. Ewell and Dennis P. Jones address in this 
paper, and which they refer to in shorthand as "Student Right-to-Know." They focus on 
developing a reasonable approach and methodology for deriving the first of these statistics, the 
general graduation rate that provides som^. useful consumer information to students and the 
public. Their analysis and approach also haii direct bearing on the second set of statistics, the 
comparative graduation rates for pai-ticular groups of students. And in a larger, less direct sense 
their argument applies as well to the reporting of crime and safety statistics, and to the demands 
for "new accountability" which these statistics are intended to address. 



^ 'Alice M. Rivlin, Systematic Thinking for Social Action (Washington, D.C.: The Prookings 
Institution, 1971), pp. 142-3. 

^Rivlin, p. 144. 
Q iv 
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Ewell and Jones demonstrate that these seemingl ..mple statistics are beguilingly complex. 
Their paper is, admittedly, detailed and technical in nature. It is not, however, narrow in scope, 
and its implications for institutional leadership, state policy and the national agenda for improving 
education are substantial. Walking us step-by-step through the dilemmas and necessary 
accommodations of constructing meaningful and useful performance information on higher 
education, this is surely an example of what Rivlin had in mind as "systematic thinking for social 
action." 

This paper provides those tiuly interested in the design and use of better information on higher 
education with an outline of definitions and procedures for analyzing and reporting on student 
progression through postsecondary education. The common template they propose goes far 
beyond what might be necessary for minimal, formalistic compliance with the Student Right-to- 
Know Act. But as Ewell and Jones imply, minimal compliance is not likely to meet the demands 
of "new accountability," and minimal compliance will not produce meaningful information to 
support real change. We need to debate and reach agreement on the details, but such a common 
template can go far to meet the information needs of institutional planners and leaders, state 
policy makers and the federal government, as well as students and the general public. Such 
information can tell us more about the changing nature of our student bodies, about our diversity 
and richness as well as the barriers and problems of student progression. This information, when 
applied in the design and implementation of policies and institutional operations, can help much 
more than quick statistics to improve the performance of education, and the outcomes of that 
education for students and society. 



Charles S. Lenth 
Denver, Colorado 



Assessing and Reporting Student Progress: 
a Response to the "New Accountability" 

Peter T. Ewell 
Dennis P. Jones 
National Center for Higher Education 
Management Systems 

Overview 

This paper reviews current approaches for traclcing student progress in postsecondary 
education, focusing on the context of emerging policy and information needs at the state, national 
sjid institutional levels and on the capabilities and limits of currently available information 
technology. The stimulus is a growing need for timely, consistent information about student 
persistence and degree-completion throughout postsecondary education. Our central argument 
is that a consistent national approach can be developed for compiling and reporting such 
irifomiation, within the limits of present technology, that can serve the neeus of all policy sectors. 

The paper consists of six main sections. First, we examine the growing requirement for 
consistent data reporting on student progress in light of some wider national policy trends. 
Second, we briefly address the methodological requirements for generating meaningful 
information about student persistence and degree-completion, and the current capacities of states 
and institutions to generate such statistics. Tlie third section contains a comprehensive discussion 
of persisting issues in this area, including such topics as who should be included in reporting, and 
how such enrollment phenomena as "stop-out," variations in credit loads carried, and inter- 
institutional transfer should be handled. In section four, we address these issues through a set 
of "minimal" recommendations for data collection and reporting. In two concluding sections, we 
discuss the implications of these ideas for future development — including possible uses (and 
misuses) of the resulting data, and how results might potentially be linked with available state 
and national "levers for change." 
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Our intention throughout is less to provide a "definitive" treatment of the informational issues 
involved in reporting student progress than to stimulate informed discussion among the various 
parties-at-interest. Our extensive experience in designing and implementing longitudinal data 
bases at both the state and institutional levels convinces us that there are no "right" answers in 
this arena; but at the same time, it convinces us that there are many ways of going wrong. We 
hope this paper proves useful in avoiding some of the latter. 

1. Introduction 

The need for information about student progress in higher education is becoming manifest 
at the national, state and institutional levels. At the federal level, immediate concerns are 
centered on effectively implementing "Student Right-to-Know" legislation, passed by Congress 
in the 1990 session, and on the need to develop consistent national indicators of student progress 
to track the achievement of National Education Goals, as outlined by the President and the 
nation's governors. At the state level, concerns are centered on the role of such infomiation in 
assessment and accountability reporting, in system planning (panicularly in system articulation), 
and in ensuring that state access and work-force needs are being effectively addressed. At the 
institutional level, longitudinal tracking information is critical for guiding effective retention and 
enrollment management, for evaluating the effectiveness of placement, remediation and academic 
prerequisite policies, as a critical component of locally-designed student assessment programs, 
and as an increasingly important component of regional accreditation. We suggest that these 
infomiation needs must be seen as part of a larger policy agenda — whiivt we have come to call 
"the new accountability." At the same time, we suggest tliat appropriate informational 
requirements should be determined in the light of differing users and uses of the resulting data. 
Together, these contextual factors suggest the shape of an appropriate higher education response. 
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Information Requirements of the New AccountiibilitY 

New kinds of information are required, we believe, because the fundamental basis of public 
accountability in postsecondary education has shifted. The case for a "new accountability" is 
rooted in changing perceptions about the public role of higher education: from a provided 
service to a strategic investment (E\/cll 1990, SHEEO 1989). This shift contains a number of 
identifiable themes, including: 

• increased emphasis on explicit linkages between higher education and achieving wider 
societal goals of renewed economic development and global competitiveness, 

• altering the focus of accountability for higher education from equitable access and 
efficient operation towsjd "return on investment," and 

• an emerging emphasis on "consumer protection" in higher education through 
publicly-available information on student outcomes and comparative institutional 
performance. 

Collectively, these themes imply the need for types of information in addition to those which 
have traditionally formed the substance of higher education's accountability. Paiticularly needed 
in this context are: 

• information about longitudinal s tudent enr ollment and perfom^ance — including 
persistence rates, graduation rates and time to completion — broken down by a variety 
of student characteristics (for example, ethnicity, gender, intended program of study, 
scholarship support, financial aid, etc.); 

• information about wha t happens to L^rad iiatt-s — including job placement and 
performance, and further participation and performance in postsecondary education — 
again broken down by relevant student characteristics; and 

• information about what graduates know and c an do — including domains of 
knowledge/content (both in general and in the student's major field), basic and "college 
level" skills .such as communications, quantitative skills, critical thinking and 
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problem-solving skills, and a variety of attitudes, values, and such "habits of mind" as 

multi-cultural awareness or community service/citizenship. 
While all three of these types of infonnation wi ultimately be required to sati->fy the 
demands of the "new accountability", the first is both the most straightfoi*ward and the most 
immediately in demand. At the same time, the technical complexities involved in generating 
»nfomiatio!i on longitudinal performance arc significantly greater tlian those involved in any 
previous type of reporting in higher education. The result is a dilemma: longitudinal 
performance statistics are difficult to generate; at the same time, they appear so transparent and 
simple that it is hard to explain to those outside postsecondary cducadon exactly wh^ they are 
difficult to generate. Given this dilemma, unusual care is required in providing the interpretive 
caveats that must bt; applied to any given performance statistic, while at the same time not 
communicating an obstructive attitude. 

Differing Inforrnalional Needs and Purposes 

Appropriate informational requirements vary across constituencies with different needs. 
Information about student progress is no exception. In developing policy at the state and federal 
levels, it is important to recognize legitimate differences in infoimational requirements and to 
ensure that these requirements ai'e developed as consistently as possible. Too often, when this 
is not the case, preventable differences in reporting methodologies between state and federal 
authorities (as v^ell as those presented by a number of external accreditation bodies) have resulted 
in the need for institutions to engage in massive and expensive recompilation of base datasets in 
order to meet the needs of multiple reporting audiences. 

Chart 1 presents a simple taxonomy of needs and purposes for information about student 
progress. Units of Analysis refer lo the prim;iry levels of aggregation at which longitudinal 
performance statistics are typically calculated. Tht v include the individual institution, statewide 
or system-level reporting and aggregate national reporting. JReportin^ Audience refers to the 
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information's prime constituency. These include higher education planners and decision makers 
at the state and institutional levels, public policy makers including legislators and lay board 
members, postsecondary education's direct "customers" such as potential students and their 
parents and the public at laj-ge. 

Intersections among these two dimensions yield quite different sets of informational 
requirements. State-level decision makers within higher education, for example, typically require 
information on system-wide rates of inter-institutional transfer and articulation for purposes of 
system-wide planning, and need information about overall degree-production rates for purposes 
of regular accountability reporting. Potential students, in contrast, require information about their 
own chances of persisting and completing a degree at a particular institution. The former 
requirements have helped to determine the content of many statewide "unit record" reporting 
systems, while the latter is the primary focus of "Student Right-to-Know" legislation. While each 
demands a different type of reporting statistic, both should be derived from a consistent set of 
base longitudinal data. A first implication, tlierefore, is that such requirements not be developed 
in isolation. Any national approach to persistence and degree-completion reporting should be 
developed consistentiy with the needs of both state and institutional decision makers. 

A second implication is more subtle. Experience with past accountability reporting has 
amply demonstrated that state and national requirements to a large extent shape institutional 
c^pacliy to produce locally useful management information. The current design of most student 
records and financial accounting systems in higher education, for example, reflects not only 
internal requirements for management information but also the need to supply standard data for 
both federal reporting and for state enrolment reporting. We expect that future requirements for 
longitudinal data reporting will have similar effects on the content and sti ucture of institutional 
data systems. As a result, anticipated national reporting should be carefully designed to induce 
institutions to develop longitudinal data bases that are consistent with current "best practice" in 
student tracking. Development of such data bases, we have found, can be both expensive and 
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time-consuming; despite their local benefits, institutions are reluctant to embark upon them in the 
absence of an external stimulus to do so. 

Experiences of the 50 or more institutions participating in the LONESTAR student tracking 
consortium in Texas are typical in this regard (Adelman, Ewell and Grable 1989). Most joined 
the consortium because thty needed a method to respond to newly-enacted state reporting 
requirements on the effectiveness of remedial instruction, and the design of the LONESTAR 
system largely reflected these reporting requirements. Now that longitudinal data bases are in 
place, however, institutions ai-e applying the resulting information effectively to improve local 
retention and student success programs. This "spillover" occurred largely because the data 
system was designed from the outset to meet both external reporting and internal management 
needs. 

Three Principles for Constructing an Appropriate Response 

Putting these elements of context together, we suggest that an appropriate national approach 
for reporting student progress should embody the following principles: 

• The resulting data system must be useful at multiple levels . Consistent with this 
principle, longitudinal data bases in structure and content should be able to meet the 
needs of individual consumers, institutional planning and accountability, statewide 
accountability and higher education system planning and anticipated federal reporting. 
Pai'ticulai- attention should be paid to the manner in which proposed reporting standards 
will positively affect overall institutional capacity to generate useful management 
information. 

• The resulting data system must be sufficiently flexible that it can operate effectively in 
quite diffe rent environments . Institutions vary significanUy with respect to academic 
policies, curricula and academic calendai-s. State systems as well vary considerably in 
their structures, and consequently in the needs that longitudinal data must meet. Each 
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of these can have an impact on the design of appropriate longitudinal data systems. At 
the same time, both institutions and states vary considerably with respect to computing 
capacity and in the structure and capabilities of their student records systems. Proposed 
approaches should be designed to allow* most institutions to generate the needed statistics 
within their current academic practices and technical environments. 
• The resulting data system must be sufficiently simple that ill types of postsecondarv 
institutions can minimally and accurately respond . As demonstrated by widely varying 
institutional responses to recently implemented "track record disclosure" reporting for 
federally-funded vocational/technical programs (Hanson 1990), a major danger involved 
in complex reporting requirements is distorted compliance. Faced with the pressure to 
report something, all institutions can be expected to come up with a number; but the 
approaches used will be so inconsistent that they are useless for both accountability and 
for guiding effective decisionmaking. Proposed requirements should be designed so that 
all can respond accurately — regardless of institutional size, type and control. . 
All recommendations advanced in subsequent sections of this paper are informed by these 

principles, and we strongly believe that any workable national approach to reporting student 

progress must be consistent with them as well. 

II. Technical Requirements and Current Capacity to Respond 

Providing meaningful infoimation on persistence and degree completion requires a 
substantially different approach to student record-keeping than does traditional enrollment 
reporting. It also requires consistent definitions for base data elements that have not previously 
been used in standard reporting. Institutional and state capacities to generate such statistics at 
present vary considerably. While current capacities to respond should not fully dictate what is 
needed, realistic planning should be informed by the kinds of student data states and institutions 
now maintain and their ability to manipulate these data. 

H 
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The standard approach to generating statistics on persistence and degree completion is use 
of a cohort methodology (Wing 1974; Ewell, Parker and Jones 1988). The unit of analysis for 
this approach is an identified "cohort" of entering students — for example, all students beginning 
their studies at Institution X in the Fall of 1991, Once identified, students remain a member of 
a gi\'en starting cohort regai'dless of their subsequent pattern of enrollmeni — including changes 
in load carried, degree objective, or interrupted attendance resulting in "stop-out" and 
re-admission. Statistics on student progress are generally calculated on the basis of the total 
number in the starting cohort. Commonly-produced indicators in both state-level and institutional 
reporting are: (a) proportion of the cohort persisting into the second year of enrollment, (b) 
proportion of the cohort persisting two or more years, and (c) proportion of the cohort completing 
a baccalaureate degree within five years of entry or an associate degree within three years of 
entry. 

In order to implement cohort-based tracking, several conditions are required: 

• The ability to regularly e xtract relevant data elements from the existing stud ent record 
system and "freeze" them at their cutrent values . Most extant institutional records 
systems are "live" — that is, when the status of a current data element changes, the new 
value replaces the old. This poses particular problems in cohort reporting because, for 
example, it may not be possible to determine a student's actual first temi of academic 
history if he or she has been readmitted. While most student records systems have the 
ability to archive files, not all institutions ai'e familiar with or use this capability. At the 
same time, most records systems aie technically limited in their ability to extract relevant 
data elements for later analysis. 

• The ability to merge rele vant data from multiple academic periods into a single file . The 
majority of cuirent student records systems — both state and iniititutional — are 
semester- or term-oriented — that is, they maintain infonnation about all students 
enrolled at the institution during a specific term. Cohort reporting, in contrast, requires 
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the ability to relate information from one term file to parallel information in the next, 
across as much as a seven-year period. Common difficulties here include changes in 
student recordkeeping procedures across terms (or in the student records system itself), 
the lack of consistent student identification numbers which are needed to match records 
across multiple terms, and the lack of the required technical tools for easily moving data 
from file to file. 

• The ability to quickly calculate relevant statistics on student progress . In contrast to 
cross-sectional descriptive information, most data elements in a longitudinal data system 
do net mean much on their own. Relevant reporting statistics instead mi-.3t be derived 
from often complex combinations of base data elements; reporting a completion rate for 
a given student population, for instance, requires knowledge of the number of 
degree-seeking students starting in a given term, the number completing programs in 
each subsequent term up to and including the current term, and a set of descriptors for 
isolating the study population. Most student records systems are designed for 
information retrieval, not statistical calculation. As a result, additional programming is 
generally required to produce longitudinal performance statistics; indeed, one standard 
practice is to use a commercial statistical software package rather than the existing 
student records system to generate such statistics, once ihe data have been extracted from 
base records. 

To meet these requirements, m ly options are available. Some institutions, particularly large 
public universities, actively maintain all past teim records in a form accessible for analysis; 
longitudinal reporting is generally the responsibility of an institutional research office with 
in-house programming staff available to design and execute the required data manipulations. A 
growing number of institutions (particularly two-year colleges) develop a free-standing set of 
longitudinal data files containing information extracted regularly from historical student records; 
once they are designed, updating and maintaining student tracking systems of this Irind is 

10 1 u 



comparatively straightforward. Most institutions, however, do not currently possess automated 
capabilities like these. Instead, they must generate such statistics by complex special 
programming within the existing student records system, or by manual aggregation ot data across 
terms. At present, we estimate that more than two-thirds of public colleges and universities and 
almost 80% of private institutions fall into this category. This does not mean that they cannot 
meet the requirements of such mandates as "Student Right-to-Know." But it does imply that 
doing so may entail a substantial additional cost. 

State capacities to conduct longitudinal studies and to consistently report graduation rates also 
vary considerably. Currently, about half the states have the capability in place to do so, though 
six of the 25 do not currently use tlMS capability (SHEEO 1991). An additional 11 states plan 
to develop a longitudinal reporting capacity within the next few years. Of this total of ,39 states, 
almost two-thirds base their tracking on a statewide unit record system, while the balance require 
regular institutional reporting using common data procedures and definitions. While this picture 
overall suggests substantial capability, state reporting systems are often limited in both 
institutional and student coverage. Some, for example, include only two-year or only four-year 
public colleges and universities, and virtually none include private and proprietary institutions. 
More importantly, they employ a wide variety of cohort inclusion criteria. Only 13, for example, 
include all credit students, 'vhile the majority track full-time degree-seeking students only. While 
these cohort inclusion criteria are sufficient to meet the reporting requirements of "Student Right- 
to-Know," they are limited in their ability to support comprehensive statewide planning studies. 

Despite limitations in current capacity, the potential state role in implementing a 
comprehensive, effective approach to student performance reporting is considerable. Even in the 
absence of a statewide unit-record data system, states can facilitate an appropriate institutional 
response by providing technical assistance and by coordinating the development of common 
definitions. The availability of a unit-record system enhances these possibilities considerably, 
as states have the potential to assume a major responsibility for reporting persistence and degree 
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completion — just as they in many cases now coordinate or directly provide centralized 
information for federal reporting. At the same time, state data bases may provide the only 
efficient method for tracking inter-institutional transfer or placement in employment. 
Applications of unit record information such as these have demonstrated that substantial 
efficiencies in data reporting can be achieved through state-level coordination and ^..k^. . iing — 
particularly for small institutions with limited technical capacity. As a result, aboui half the 
states expect to play a role in "Student Right-to-Know" reporting, and an additional 1 1 would like 
to develop the capability to do so (SHEEO 1991). 



111. Persistent Issues in Reporting Student Progress 

Cohort methodologies have been repeatedly employed in research studies aimed at 
determining the reasons for student non-persistence and success. Less extensively, they have 
been used to generate statistics on retention and degree completion for purposes of public 
reporting — particularly for identified groups such as minorities and student athletes. Whatever 
the purpose, however, a number of definitional issues hinder the ability to carry out such studies 
systematically. While the "Student Right-to-Know" legislation suggests some procedural 
resolutions to these issues at the federal level, these are far from definitive. A comprehensive 
approach useful at the institutional, state and national levels must recognize the complexity of 
these issues and suggest appropriate definitional resolutions. 

This section of the paper is intended to present these issues systematically, examining the 
merits and drawbacks of alternative approaches. A subsequent section presents a set of 
procedural recommendations for addressing these issues. 



Definitional Consistency 

Consistent definitions are required for base data elements used in reporting, for procedures 
used in constructing cohorts, and for calculating performance statistics derived from base data. 
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While the first and third are familiar issues in public reporting, the second is more complex and 

has not previously been encountered in public reporting. 

1. Data Issues. Base data requirements for longitudinal student tracking are of two kinds. 
First, student descriptors are required to identify populations of interest; such variables will 
usually be used to break down overall performance results by group — most commonly, for 
example, by gender or by ethnic population. Descriptors of this kind are generally of three 
kinds (Ewell, Parker and Jones 1988): 

• Demographic descriptors cover various student background characteristics that describe 
recognizable population groups; among the most common are gender, ethnicity, date of 
birth, geographic origin and physical disability. 

• Educational background descriptors provide information on educational aptitude and prior 
attainment; examples include high school attended, high school performance, aptitude and 
basic skill test scores, and previous postsecondary educational experience and attainment. 

• Initial enrollment status descriptors provide information about a given student's status 
when first entering a college or university; the most common include first-time freshman 
vs. transfer student, average credit load carried (or "full-time/part-time"), time of 
attendance (for example, "day/evening"), special vs. regular admissions smtus and initial 
major or program of study. 

Many of these descriptors — particularly in the first category, jire already defined at the 
national level. Many, however, are not. 

A second data requirement for longitudinal uacking is for performance and outcome data. 
In standard cohort methodologies, typical data elements include credits enrolled for and 
completed, academic performance, program/major changes, and program completion within 
a given time period. Generally, information is collected for each student separately for each 
term in the tracking period, and is later aggregated across terms to yield overall performance 
statistics for various types of students. Such methods have served individual institutions well 
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in calculating meaningful information on student progress within their own contexts. When 
aggregating across institutions and states, however, a number of additional definitional 
problems arise. Among them are: 

• Definitions of "persistence ." Under most cohort approaches, students are held to be 
actively enrolled if ' .ley register for at least one credit during the term of record. Some 
institutions, however, continue to can7 students as "enrolled" in a given term when they 
are not taking classes — often in connection with a planned year off or abroad. For 
national persistence reporting, such inconsistencies will need to be resolved, though for 
the most part, current definitions for counting "enrolled" students in fall enrollment 
reports for the federal Integrated Postsecondary Education Data System (IPEDS) suggest 
a point of resolution. 

• Variable academic calendars (and resulting term structures). As noted above, standard 
cohort methodology requires term-to-term tracking of student progress. Nationally, 
colleges and universities use a variety of academic calendars that differ with respect to 
the number of terms in an academic year (semester, trimester and quarter), with respect 
to start and end dates for each term, and with regard to how summer and "intersessions" 
are handled. Res. . ; this difficulty virtually requires annual rather than term-oriented 
reporting of persistence, but this resolution raises additional issues: for example, is a 
student "persisting" if he or she enrolls for only one term in a given academic year, or 
must enrollment take place in aU regular terms? 

• Inclusion of college-level courses only in calculating, persistence . Many students arrive 
at college deficient in basic skills, even though their objective is to earn a degree. 
Through placemfmt testing, such deficiencies may be detected and students required to 
remediate deficiencies through non-collegiate coursework. Should enrolling for such 
coursework be considered in determining "persistence" or should only college-level work 
be reported? 
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In sum, in order to provide meaningful longitudinal information and minimal 
compliance with requirements such as "Student Right-to-Know", standard definitions will 
be required for a number of new data elements not now reported to the federal 
government or, in a standard way, to any other multi-institutional agency. 
2. Cohort Construction and Inclusion . Beyond base data definitions, consistent longitudinal 
information requires procedures for how student cohorts are defined and who is to be 
included in them. This is an issue which has not be previously faced in standard reporting 
and which requires particular care. As athletic persistence reporting has shown, even small 
variations in cohort construction ."-^d inclusion criteria can yield enormous differences in 
calculated degree-completion statistics; similar lack of standardization has frustrated most 
attempts to compare the results of retention research studies nationally. 
In order to arrive at meaningfully comparable results, we believe that at minimum the 
following definitional questions must be addressed: 

• What is an appropriate start date / Standard cohort methodologies use two quite different 
methods for determining the "first term of enrollment" for a given student. One is 
behavioral: students are included in the cohort that corresponds to the term of their first 
credit enrollment, regardless of load carried or degree intention. The second is related 
to matriculation: students are included in the cohort that corresponds to the term in 
which they first matriculated as degree-seeking students, regardless of prior attendance 
at the institution. Both are valid definitions of "stun date" for different purposes. While 
the language of the "Student Right-to-Know" legislation appears to require a cohort 
definition based on matriculation ("full-time degree-seeking students"), procedural 
implications remain to be clarified. Behavioral cohort definition has the advantage of 
conceptual clarity, but can result in anomalies of classification: a student who regisiors 
for one personal interest class at a community college and two years later begins a 
degree program at the same institution, for instance, is placed with the cohort that 
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corresponds to his or her first class. But a matriculation- based cohort definition has 
equivalent drawbacks. Students may declare a given degree well after they have begun 
earning credit towaid it, thus gaining a "head start" on their counterparts who declared 
a degree intention on admission. At the same time, students may earn degrees in 
sequence — particularly in articulated programs; do they become baccalaureate 
degree- seekers on entry, or only after they have completed an associate degree? 
What is the appropriate unit of analysis for cohort construction ? Most cohort 
methodologies take the individual institution as the unit of analysis. Students are 
included in a given staiting cohort on the basis of when they enter the institution, not on 
the basis of when then began their postsecondary studies. Nationally-based cohort 
studies such as NCES' National Longitudinal Study and the High School and Beyond 
Study use a "systemwide" unit of analysis: students remain in a given starting cohort 
regardless of how many postsecondary institutions they attend. State systems of higher 
education may also occasionally construct system-level cohorts that track students from 
the institution at which they began their studies to the institution at which they earned 
a degree. Such studies are paiticulurly useful to assess transfer performance in 
articulated two-yeai' and four-year public systems. Again, there is no right answer in 
developing a consistent national reporting methodology. The obvious intent of such 
reporting requirements as "Student Right-to-Know" is to provide institution-level 
statistics. But this must be accomplished in a manner that reflects the probability of a 
given student's attending multiple institutions, and persistence and degree-completion 
statistics must be calculated in a manner that does not aitificially truncate the time 
required to earn a degree for students who attend more than one institution. 
What is the appropriat e wav to handle prior credit earned ? This question is closely 
related to the one above. Increasing numbers of students enter colleges and universities 
as transfer students, and begin with advanced standing at the receiving institution. Their 
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progress cannot, therefore, be legitimately compaj-ed directly to that of first-time 
freshmen. Most existing statewide persistence and degree-completion reporting obviates 
this issue by calculating results for first-time freshmen only. If transfer students are to 
be included, however, two options are generally followed. As above, a first alternative 
is behavioral: u-ansfer students are placed in the cohort that corresponds to their first 
actual tenn of enrollment at the institution, are identified as transfers, and their total 
number of transfer hours recorded; this procedure allows actual persistence rates to be 
calculated for transfers, but it may distort overall time-to-completion stanstics if transfers 
are not separated analytically from first-time freshmen. Alternatively, transfer students 
may be placed with the cohort that corresponds to their class standing on admission; this 
procedure results in reasonable "bottom line" statistics when all students are aggregated, 
but significantly distorts what transfer students are actually doing. Requirements such 
as "Student Right-to-Know" suggest a behavioral approach to tracking transfers, but also 
appear to require separate reporting for first-time freshmen. 

Taken together, issues of definitional consistency will probably prove the most 
challenging issues for standard reporting. Once a coherent approach is developed, the 
answers to many additional issues will also be partially determined. 

Student Patterns of Attendance 

The primary reason why issues of cohort definition are so important is that students are 
increasingly interacting with postsecondary institutions in a "non-traditional" fashion. So long 
as an expected four-year pattern of full-time, continuous attendance is assumed, most of the 
problems associated with measuring persistence disappear; indeed, this is a major reason why 
many institutions and states have confined themselves to tracking full-tim^, first-time freshmen 
only. But this is a highly unreasonable assumption. Once it is relaxed, it is apparent that there 
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are many reasons why students can fail to complete degrees within an "expected" time period — 
not all of which are within the institution's purview or control. 

In general, issues related to student patterns of attendance are of tv;o types. The first has to 
do with enrollment behavior at a single institution — whether a student chooses to attend 
full-time, to stop-out, or whether he or she experiences academic difficulty. The second has to 
do with the increasingly frequent practice of attending more than one institution. 

1. Varying Enrollment Behaviors . Public disclosure and reporting requirements such as 
those contained in the "Student Right-to-Know" legislation assume that statistics on student 
persistence and degree-completion can legitimately be used for comparing institutional 
performance. But many factors are responsible for obtained differences among institutions 
on such measures, not all of which are under an institution's control. A key problem, 
therefore, is to distinguish among those outcomes resulting from actual differences in 
institutional performance and those that are a natural consequence of serving a particular 
student body. 

The language of the "Student Right-l^-Know" legislation, for example, bases graduation 
rates on the proportion of entering students who "complete degrees within 150% of the 
normally allotted time period." Those who fail to complete within this time period, however, 
may do so for many different reasons: 

• Thev drop out voluntarily and in good academic standing . Students who terminate their 
enrollment at a given college or university are normally seen as institutional failures; 
"retention" in this sense is viewed as almost exclusively positive. But students can 
withdraw voluntarily under two quite different conditions — having met their personal 
objectives, and having not met those objectives. Many students enrolling at two-year 
colleges, for example, do so largely for reasons of obtaining needed job skills to obtain 
or to upgrade employment. If these objectives can be fully met without earning a 
degree, the degree itself is of little importance. Sirnilariy, as many as two-thirds of 
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community college transfer students who will subsequently enroll in a four-year college 
in pursuit of a baccalaureate do so without having previously completed an associate 
degree. In neither of these cases can withdrawal be seen as institutional failure. 
Substantial experience has demonstrated that it is not safe to assume that such students 
will be classified as "non-degree" in student registration systems and subsequently not 
included in standard cohort definitions. For a variety of programmatic reasons, for 
instance, many such students are present in the ranks of community college 
"degree- seekers." 

They fail a cademically . Most studies indicate that a relatively small proportion of 
students who do not continue their studies at a given college fail outright. Certainly, 
academic difficulty is a primaiy reason for early voluntary withdrawal, but it is far from 
the only reason. From the standpoint of institutional performance, moreover, academic 
failure rates are difficult statistics to interpret: does a higher- than-average failure rate 
connote an institution that does not sei-ve its customers well, or an institution with 
rigorous academic standards? Regardless of the answer to this question, separately 
reporting the proportion of students who fail to complete degrees for reasons of academic 
performance is cleiu-ly needed if a major intent is to provide useful information for 
guiding consumer choice. 

They attend part-time. Obviously, students may take long times to complete degrees if 
they do not carry full loads. Reporting requirements such as "Student Right-to-Know" 
take this into account by calling only for the calculation of persistence and 
degree-completion rates for full-time degree-seeking students. But experience has shown 
that students may significantly vaiy their credit loads from term to term over time. In 
a recent study of two-year degree completion in Indiana, for example, only 26% 
degree-seeking starters maintained full-time loads throughout all regular terms of the 
tracking period. Equally obviously, the number of credits carried is a student choice; it 
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is a characteristic of the student constituency served far more than it is a factor that 
signifies institutional performance. As a result, degree-completion reporting should allow 
clear identification of the extent to which low completion percentages within a given 
fixed time period ai'e a direct consequence of legitimate student enrollment choices. 
Thev "stop in" and "stop out ." Students may also voluntarily interrupt 'heir enrollment 
for several terms at a time, even though they carry full-time loads for the terms that they 
do attend. The consequences for the time required to complete a degree are identical to 
those associated with part-time attendance and, like part-time attendance, are more 
descriptive of the clientele served than of the institution's performance. Moreover, such 
students are very likely to begin their studies as full-time degree-seekers, and thus be 
included in an initial tracking cohort. As above, additional information is needed to 
determine the rate of interrupted enrollment present at any institution in order to judge 
fairly time-to-completion within a fixed time period as a useful piece of consumer 
information. 

Thev cannot enroll for the proper required courses . Increasingly long times typical for 
completing baccalaureate curricula are also partly the result of escalating requirements. 
Many professional baccalaureate programs in fields such as engineering, architecture and 
education, for example, demand well in excess of the 120 semester credit hours typically 
required to earn a baccalaureate degree. At the same time, students may complete more 
than the number of hours necessai^ to graduate because they are blocked from 
registering in key courses needed in their majors or to complete the general education 
portion of their degrees. This is an increasingly salient problem at large universities, 
where oversubscription for needed courses is routine. Both these reasons for extended 
degree-completion are cleiU"ly factors of institutional performance, and should be so 
recognized in reporting. 



Because of these many variations in student attendance pattenic, meaningful reporting 
of persistence and degree-completion for purposes of institutional comparison demands 
additional contextual statistics that can help separate factors of student choice from 
institutional performance. At a later point, we suggest a number of such indicators as 
part of our recommended reporting procedures. 
2. Attending Multiple Institutions . A second variation in enrollment patterns that can 
significantly affect the reporting of persistence and degree-completion rates is attending 
multiple institutions. Increasingly, students are earning credit at more than one institution 

in either a planned or unplanned way. Recent studies in California, for instance, suggest that 

» 

almost half of those who earn a baccalaureate at a California public institution began their 
studies at one of the state's two-year colleges. Obviously, students who transfer credit into 
a given institution attain advanced standing, and can therefore be expected to earn degrees 
more quickly than those who do not do so. As a result, their inclusion in average 
time-to-completion statistics will severely skew results for institutions with large numbers of 
transfer students. This is a major reason why many persistence reporting systems track and 
report on first-time freshmen only. 

At the same time, some aiticulated state systems of public postsecondary education have 
recently undertaken studies that track persistence and degree-completion for the system as a 
whoie. Times to baccalaureate, for example, can in this manner be calculated as a whole — from 
initial point of enrollment at a two-year college in a transferable program to eventual completion 
of a baccalaureate at a four-year institution. Arguably, this kind of information provides potential 
"customers" with what they really need to know: "how long is it actually going to take me to 
complete my degree objective, no matter how many institutions in the system I end up 
attending?" Because they are oriented toward single-institution reporting, requirements such as 
"Student Right-to-Know" cannot provide this information. Taken together, these issues reinforce 
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the concept of separating persistence reporting for uansfers and for those whose entire careers 
as students are spent at the same institution. 

Addressing systematically tne issues associated with varying patterns of student attendance 
is critical for meaningful reporting of information on student progress. In most cases, moreover, 
the clear requirement is for additional information to be reported that allows recipients to assess 
the degree to which reported rates are due to institutional context or are due to actual differences 
in institutional performance. 



Some Particular Problems with Reporting Program-Level Statistics 

Considerable interest in reporting student progress centers on the feasibility of producing 
persistence and degree-completion statistics for individual academic majors or programs of study. 
In addition to the variations in overall student patterns of attendance noted above, several special 
difficulties are associated with any attempt to track student progress b/ field of study. At the 
same time, some approximate indicators -can be quickly calculated that, if interpreted cautiously, 
provide some useful infonnation on major field performance. 

1- Variations in Major Field Enrollments . While in principle possible, tracking student 

progress within identified majors is extremely difficult in practice at the undergraduate level. 

Particular difficulties associated with major field tracking include: 

• Varying da tes of program declaration . While many undergraduate students enroll for or 
are admitted to a particular program or major at entry as freshmen, the majority do not. 
Instead, they declare and are accepted by programs at some point within their first two 
years. Statistics on "student major" cajried in most student registration data bases are 
therefore self-reports of the student's intent rather than actual classifications of status. 
Large numbers of current students, moreover, remain "undecided" until relatively late in 
their academic caieers. Because many academic programs require nc independent 
admission and are based on completing a defined aggregation of courses, students may 



"major" without official declaration. Indeed, at some institutions it is not unknown for 
students to submit a degree petition and a declaration of major at the same time! 

• Changing majors . Students also frequently change from one major to another in the 
course of an undergraduate career. Because of vjiriations in the structure of degree 
program requirements, such changes are often roughly predictable (for exarr.ple, students 
are far more likely to change from physics to sociology than the reverse), but they 
nevertheless occur with a great deal of variability. 

• Simultaneous enrollment in more than one program . Typically encountered patterns here 
include "double major" (majoring in two fields at the same time), "major/minor" 
(enrollment in two fields simultaneously with varying and non-equivalent completion 
requirements in each), and sequential completion of more than one major before 
graduation. 

2. Some Feasible but Limited Approaches . Given the above difficulties, meaningful 
program-level statistics on student progress are extfemely difficult to generate. Two methods 
used in past studies, however, are worth further exploration. Both are indicative of program 
major perfonnance, but both also have substantial limitations. 

• A prospective method is based on initial degree field declaration. The relevant 
perfonnance statij:tic is the proportion of a stai ting cohort beginning a particular program 
who in fact completed that (or another) program at the institution within a given time 
period. This method is limited as above by the difficulty of establishing a consistent 
point in time for program declaration. To be accurate, the time taken to earn credits 
counting toward the major field degree, but earned prior to program declaration, should 
also be taken into account. Because time of declaration is uncertain, it may be 
impossible to calculate time to degree, though the proportion of a starting cohort that 
actually earns a degree may be derived. 
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• A retrospective method is based on the field of the degree actually granted, regardless 
of whether or not the student began study in the same major field. Under this method, 
all students receiving degrees in a given field in a given term are identified, together with 
their first active terms of enrollment at the institution, and an average elapsed time 
calculated. While this method will easily yield time-to-degree statistics, it cannot 
produce graduation rate by field, because the proportion of students switching out of the 
major before completion is not known. Both methods have been used in past studies as 
indicators of programmatic effectiveness. 
Overall, we conclude that systematic reporting of student progress by individual fields of 
study is not feasible. Certain professional, occupational, and technical fields where students are 
admitted directly to programs of study and tend to pursue them in isolation may be exceptions, 
however, and can be tracked. This includes, for example, most two-year vocational/technical 
programs. Fortunately, many priority programs for state and national manpower development, 
such as engineering, technical, health-related or other scientific fields, often fall into this 
category. Most experience indicates that, because of sequential curricula that require an early 
start on prerequisite courses, students tend to transfer out of such programs after initial 
declaration for more frequently than they transfer into them. As a result, prospective tracking 
in such programs may be feasible. 

Some Problems with the Time Period Allowed for Degree Completion 

Consistent reporting of degree completion rates requires a standard elapsed time period. It 
has become customary in student retention studies at both the institutional and state levels, for 
instance, to report is a summary statistic the proportion of a starting cohort completing a 
baccalaureate degree within five years. Some proposals for national reporting similarly suggest 
reporting the proportion of starters who complete within "150% of the time normally allotted for 
program c jmpletion." While conceptually attractive in producing a single indicator, however. 
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limiting completion reporting to a single benchmaik such as "150% of nornial" time elapsed has 
a number of drawbacks. Among them are: 

• Varying curricular re quirements among programs . As noted previously, many 
baccalaureate degree programs require more than the typical 120 semester credit hours 
(or equivalent) for completion. While it is assumed that published catalogue descriptions 
will be used to determine "normal" time to completion, it must be recognized that such 
descriptions do not always do so. 

• Varying conceptions o f "normal progress" for good standing . Institutions vary 
considerably in the maximum time limit (if any) that they place on students for earning 
a particular degree. While this may or may not affect national reporting standards, it 
may influence student perceptions of what constitutes "normal" progress. If only 
students who initially attend full-time are tracked (as suggested by "Student Right-to- 
Know"), this may be less of a problem, as such limits are primarily meant to apply to ^ 
students who stop out for long periods or v.ho enroll for exuemely low credit loads. 

• Inclusion of colleoe-lev el coursework only . Again as noted, many students begin 
postsecondary study deficient in basic skills, and as a result are placed in non-collegiate 
coursework until such deficiencies are remediated. If terms in which exclusively 
non-collegiate work is taken are included in the "150% of normal" time period, 
institutions that enroll substantial numbers of undeiprepared students will show deflated 
comparative graduation rates — even thoug.i sei*ving such students is their primary 
mission. On the other hand, not including pre-collegiate work in the reportable time 
period may give a distoiled impression of how long underprepared students will actually 
require to complete degrees. 

As noted, standard practice in retention reporting requires that "benchmark" statistics be 
calculated for degree completion within fixed time periods. Both institutional and state practice 
in longitudinal reporting, however, typically involve reporting at several points in time (five, six. 
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and seven years after starting, for example), and also leport an overall average time to degree 
completion. Multiple statistics of this kind have proven far more informative than comparative 
statistics tied to a single fixed time period. 

Follow-up After Completion 

Though not strictly within the realm of student progress reporting, demands for information 
about what happens to students after leaving postsecondary education are also growing. For 
example, it is suggested by the "Student Right-to-Know" legislation that institutions generate 
postgraduate placement statistics for several purposes — both directly, in the form of job 
placement rates for completers, and indirectly, in order to determine if withdrawing students have 
continued their postsecondary studies "in programs for which the institution provided substantial 
preparation." Because follow-up is often undertaken in conjunction with student tracking, we 
believe that it is appropriate here to briefly touch upon some of its compelling methodological 
issues. 

1 . Some Unresolved Issu es, While an extensive analysis of the problems encountered in 
conducting follow-up studies in postsecondary populations is beyond the scope of this paper, 
a number of issues are sufficiently important that we believe they should be raised in 
conjunction with reporting student progress. They include the following: 
♦ Within what time period ? Consistent reporting of both placement and re-entry into 
postsecondary study in "related" fields requires that a standard time limit be established 
as a cut-off. For entry-level job skill programs, the usual period used is one year or less. 
But the appropriate follow-up period is less clear for programs of study that are not 
explicitly intended to result in entry-level employment. Similarly, what time period 
should be allowed for re-enrollment in a program for which the institution provided 
''substantial preparation?" Again, one year seems a reasonable starting point, but in most 
cases earned credits will remain transferable for considerably longer. 
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• Should non-completers also be included ? Most established institutional follow-up 
systems are focused on program completers only. But consistent reporting may 
reasonably require follow up of those who withdraw without completing a program as 
well. First, for many students, program completion is not a primary objective. In 
occupational/technical programs, the primary objective may be obtaining a job, and if 
this can be accomplished before the program is completed, students may cease to attend 
because their objectives are met. The same is true for transfer students at community 
colleges who intend to complete a baccalaureate elsewhere. Both these conditions 
suggest that both program completers and those who leave the institution without 
completing a program be included in any follow-up reporting effort. 

• What d o es "related field" mean ? For both post-graduate employment and re-enrollment 
in postsecondary study, a distinction is generally made between those who accomplish 
either of these activities in an area demonstrably related to their prior postsecondary field 
of study and those for whom this is not the case. If this distinction is to be made in 
standard reporting, hov^ever, consistent definitions mi zi he developed for both the 
particular occupational classifications deemed "directly related" to any given field of 
study, and for the particular areas of postsecondary study that a given prior 
postsecondary program provides ' substantial preparation." In the occupational areas, 
some work has already been accomplished in defining "related" fields for federal 
reporting under the Vocational Education Data System (VEDS) and federal "Track 
Record Disclosure" reporting. In the academic area, however, standard definitions of 
prior "substantial preparation" will need to be developed, perhaps guided by a study of 
current standards for U-ansferable credit and articulation. 

• Should a c ohort or cross-sectional approach be ii.sed ? Standard practice in conducting 
post-enrollment follow-up studies takes as its reporting base all those who complete 
programs (or who withdraw) within a given academic year. Under this approach, 
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however, the students surveyed may have begun their studies at quite different points in 
time. Resulting statistics ai'e thus based on a different population than the cohort base 
used for calculating rates of persistence and degree-completion; placement rates obtained 
in this manner thus cannot be directly compared to statistics on student progress. Using 
a cohort methodology in conducting follow-up studies after program completion or 
withdrawal, however, entails additional effort in data collection; not only must the 
students who leave be identified, but their start date must also be determined. At the 
same time, assembling follow-up statistics (such as job placement rates or continuing 
postsecondary persistence) by cohort i? difficult because information collected 
cross-sectionally (e.g., a survey of last year's graduates) must be re-allocated across a 
number of starting cohorts on a case-by-case basis. In general, it must be expected that 
most institutions will continue to use a cross-sectional approach to post-enrollment 
follow-up, and appi'opriate standard reporting procedures should be designed with this 
likelihood in mind. 

2. Possible Methods for Follow-up . The primary approach used by colleges and universities 
to determine rates of job placement and further postsecondary study is based upon mailed 
surveys. Alternatively, some institutions conduct telephone surveys. Although some 
institutions have obtained usable data from this method, response rates to both kinds of 
surveys are often disappointing, and can result in highly unreliable information. 
If standard statistics ai'e to be meaningfully generated, acceptable parameters for response 
rates should be established. In some existing cases (for example, in the program review 
processes of community college occupational/technical programs in Florida and Maryland), 
response rates are reported together with the statistics themselves. This approach allows 
"consumers" of the report the opportunity to roughly gauge the reliability of obtained placement 
and postsecondary persistence rates. An alternative is to establish a minimum standard response 
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rate for reporting. Experiences with such studies, for example, suggests that response rates of 
below 25% are unlikely to yield reliable infonxiation. 

Actively exploring alternatives to survey-based follow-up reporting may also prove fruitful. 
Re-enrollment in another postsecondary program after program completion or withdrawal, for 
instance, is easily determined electronically in states that maintain a unit-record database that 
includes information drawn from all public colleges and universities. As noted above, nearly 40 
states currently maintain or are developing such systems, and they have been used in at least 10 
states to produce reliable statistics on interinstitutional transfer and graduation rates. A drawback 
of such systems, of course, is that they do not contain records drawn from private institutions. 
Another emerging approach is the "electronic transcript" now being developed by several college 
and university consortia. Under this method, enrollment and performance data can be transmitted 
from one institution to another in machine-readable form. This method is also being explored 
as an alternative to surveys for collecting more complete information about post-enrollment 
activities and performance. 

In the realm of job placement, similai- alternatives appear promising and are being explored. 
In Florida, for example, statistics on cuiTent employment in field and average earnings can be 
automatically calculated for both program completers and non-completers by matching college 
records electronically with statewide employment records. Similar projects arc now under way 
in Colorado, Texas, Oregon and Washington. This approach appears capable of generating 
employment statistics on over three-quaiters of an identified postsecondary population, compared 
with the often less than a third who complete a follow-up survey 

Again, full discussion of post-completion follow-up is outside the scope of this paper. A 
comprehensive approach to reporting institutional performance, however, demands such 
information, and student progress reporting should be designed to conform as fully as possible 
to the eventual need for such reporting. 
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IV. Some Recommendations for a Basic Reporting System 
Given the complexity of the issues noted above, many resolutions are possible. In this 
section, we present for discussion one set of possible directions. In developing these 
recommendations, we have attempted as fully as possible to follow the principles noted earlier. 
Any approach to student progress reporting must serve multiple constituencies (federal, state and 
institutional), must be sufficiently flexible that it can be implemented in many technical and 
academic contexts, and must be simple enough that meaningful information can be produced by 
all institutions. Our intent here is to provide a concrete starting point for future discussions, not 
to advance a set of definitive solutions. But we firmly believe that a concrete template is needed 
with respect to methodologies, data elements, and reportable statistics if fruitful further discussion 
is to proceed. 

Our recommendations are presented in three parts. First, we advance some broad 
methodological procedures with respect to cohort construction and analysis. Secondly, we 
suggest a number of standard reporting statistics and how they might be calculated and 
interpreted. Finally, we propose a minimal base dataset to produce the required statistics, and 
point out the areas where additional data element definition will be needed. 

Recommended Procedures for Cohort-Building and Analysis 

As noted, both institutions and state systems have in the past used many approaches in 
reporting student progress. Vaiiations occur with respect to how longitudinal uacking cohorts 
are built, who is to be included in such tracking, and how these data are to be reported and 
analyzed. Our recommendations in this arena are presented below. 

1 . A Cohort-Based Methodology Based on Start Date at a Given Institution . We 
recommend that any comprehensive system be founded on establishing entering conorts on an 
institution-by-institution basis. While states may find it useful to occasionally establish and track 
entering cohorts for a public higher education system as a whole (for example, in examining 
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patterns of interinstitutional transfer), and the federal government may find it useful to continue 
looking at overall collegiate completion rates through such national cohort studies as the National 
Longitudinal Study and High School and Beyond, the primary focus of attention for 
comprehensive student progress reporting should be the individual institution. This requires tiiat 
longitudinal cohorts be established for each postsecondary institution. We strongly believe that 
the resulting management information will be as useful lo institutional decisionmakers as it is for 
providing external consumer information or accountability. As a result, a major intent required 
of progress reporting design should be to encourage and support "best practice" in constructing 
and using longitudinal data systems at the institutional level. 

We must emphasize that this recommendation should in no way discourage states and 
systems from adopting a collective, comprehensive annroach to reporting progress. Important 
efficiencies in institutional cohort tracking, for example, can be achieved through use of a 
centralized unit-record system. In such cases, the software and procedures needed to create and 
update tracking cohons can be standardized, and the resulting files shared witJi institutions; at 
the same time, unit-record airangements provide the system with the capability to easily conduct 
articulation studies of interest to each institution. States can also play an important technical 
assistance role in helping institutions to build the required local tracking capability. In Texas and 
Hawaii, for example, institutional tracking systems were developed in common for the state's 
two-year colleges under a consortia arrangement funded and coordinated through a state system 
office. 

We further recommend that starting cohorts be established for earh fall term, and should 
include all incoming students who tirst register for credit at the institution in the fall term of the 
academic year. While some institutions may choose to establish starting cohorts for terms other 
than fall, academic calendiu-s vary sufficiently that uniform reporting of cohort statistics is 
impossible. At the same time, standard fall temi reporting is already established at both the state 
and federal levels, and existing definitions and inclusion procedures for IPEDS and similar 
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systems can be followed. We recommend against establishing starting cohorts that include all 
students entering the institution in a given academic year, regardless of term, on the grounds of 
excessive complexity. Most institutions maintain data on a term-by-term basis, and aggregating 
starters across terms can be technically difficult. At the same lime, the resulting statistics 
(particularly persistence statistics) are less useful locally than ai'e term-based statistics. We 
recognize that this recommendation means that not all starting students will be tracked, and that 
student progress statistics will as a result be "incomplete." But we believe in this case that 
procedural simplicity and tlie utility and interpretability of the resulting information outweigh 
exhaustiveness. 

We additionally suggest the possibility of including prior summer starters in each fall cohort. 
Students are often admitted for a given fall term, but physically attend the institution for the first 
time in the summer before. Moreover, summer starters often contain populations of particular 
interest for tracking, including athletes and students with assessed basic skills deficiencies who 
require remediation. We emphasize, however, that new definitions are needed that 
unambiguously specify if all, or a subset, of prior summer starters should be included in a 
standard fall tracking cohort. 

2, Include Matriculated Detzree-Seekers Only . Because a primary intent of student progress 
reporting is to determine comparative rates of degree-completion, we recommend that only 
degree-seeking students be included. All types of degree-seekers should be initially distinguished 
(e.g., associate vs. baccalaureate degrees) according the "catalogue time" required for their 
completion. This recommendation is consistent with but broader than the inclusion criteria of 
the "Student Right~to-Know" legislation, which specifies a further limitation of full-time students. 
We believe that meaningful local tracking systems must include aU students regardless of initial 
credit load, and full-time students can be easily isolated for special analysis and reporting if all 
starting degree- seekers are initially identified and included in the cohort. Including all starting 
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degree-seekers, moreover, obviates the difficulty of uneven full-time attendance; any subset of 
students, regardless of enrollment pattern, can be tracked. 

We funher recommend that cohort assignment should wherever possible be based upon the 
point at which a student first began earning credit toward a degree. Behavioral cohorts 
containing all types of students are more useful to institutions than more exclusive approaches, 
and "matriculated" students can be flagged and isolated for reporting in a longitudinal data base. 
The main reason for this recommendation is that students may begin earning credit toward a 
particular degree long before they officially declare a degree intention. We suggest two potential 
ways to deal with this situation. First, an earned-credit limit might be established, beyond which 
students are considered behaviorally to be "degree-seeking," regardless of declaration. The Texas 
Academic Skills Program (TASP), for example, requires students to be assessed for basic skills 
as "degree-seeking" if they have amassed fifteen or more credits of degree-related work. 
Alternatively, students mielit be assigned directly to the cohort that corresponds to the term in 
which they first enrolled for a course that can be applied toward a degree. The first approach 
would increase degree-completion rates by eliminating legitimate (but undeclared) degree-seekers 
who drop out in their first few terms; the second would artificially decrease degree-completion 
by including students who registered for degree-credit courses who have little or no intention of 
actually completing a degree. 

3- Ter m Irackinti and Fall Reportinii . We recommend that all academic activity be 
recorded and tracked on a term-by-term basis at each institution, but that standaid persistence and 
completion reporting be established on the basis of fall term "snapshots" only. This 
recommendation simultaneously recognizes the utility and simplicity of term-based methods at 
the institutional level and the fact that variations in academic calendars render common 
term-based reporting impossible. We believe that institutions would be ill-served by developing 
a .ipecialized tracking systein to meet the exclusive need of federal reporting. At the same time, 



fall term snapshots of cohort progress provide the only feasible method for comparing 
perfomnance across institutions on a consistent basis. 

We further recommend that the tracking period not be artificially constrained to "150% of 
the normal time period" required for degree completion. While "Student Right-to-Know" 
suggests reporting statistics on this basis, we believe that institutions are not well served by 
artificially cutting off the tracking period because external reporting does not require statistics 
beyond six years. Reporting requirements should include both an average time to degree 
completion and, if feasible, degree completion rates for multiple elapsed time periods (for 
instance five, six and seven years after initial matriculation). We believe that this more flexible 
approach will both foster better institutional use of the resulting information and will provide 
better "consumer information" than a performance statistic collected at a single point in time. 

4. Distinguish First-time Freshmen from New Transfers . To minimally resolve the 
difficulties of prior credit discussed in the previous section, we recommend, a) that new freshmen 
and new transfers who enter the institution in the same term should be included in the same 
starting cohort, and b) that all progress reporting statistics for these two groups should be 
compiled and presented separately. The reasons for tliese recommendations are several. First, 
effective management requires information about the enrollment patterns of both freshman and 
new transfer populations; for many institutions, transfers constitute over half the incoming student 
body each term. Secondly, we recognize that best practice at the institutional level requires 
constructing actual behavioral cohorts rather than assigning incoming li-ansfers to the cohort that 
corresponds to then- expected "graduating class"; only a behavioral approach will accurately 
reflect what is actually happening at a given institution. Finally, we do not believe that an 
appropriate method can be devised to meaningfully aggregate statistics across transfer and new 
freshmen populations or indeed, across new transfer students who bring with them varying 
amounts of transfer credit. Assigning new transfers to an appropriate "starting class" based on 
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;he amount of credit transferred will distort the fact that incoming transfers may have taken 
varying amounts of time to amass these credits, as have the freshmen starters with whom they 
are grouped. At the same t^me, producing "bottom line" degree completion statistics for a 
behavioral cohort as a whole will artificially decrease times to completion by mixing together the 
shorter average tiines requiied by transfer students to complete their degrees. 

For similar reasons, we recommend separate reporting lines for categories of new transfers, 
broken down by the number ol credits successfully transferred or the class standing awarded to 
the student on initial enuy (e.g., sophomore, junior, etc.). If numbers of transfer credits are used, 
these should be translated into academic year equivalents at the receiving institution (e.g., 
students transferring one year of degree credit or less, two years of degree credit, etc.). Again, 
the reason for this recommendation is to allow inclusion of all entering students with their actual 
starting cohorts without producing misleading persistence and degree-completion statistics for the 
cohort as a whole. At the same time, we believe that separate reporting will yield better 
consumer information for potential students because incoming transfers can more precisely assess 
the institution's past perfonnance for students transferring similar numbers cf credits. 

5- Report Students Who Pers ist at Another Institution Seoaratelv . We recommend that all 
statistics on persistence and degree-completion be compiled on the basis of enrollment at a single 
institution. This is consistent with current practice in student tracking at most colleges and 
universities and yields straightforward easily-inteipretable statistics. This recommendation is at 
variance with current practice in reporting the persistence of student athletes under NCAA 
guidelines, and with proposed reporting under "Student Right-to-Know." In both latter cases, 
students are to be counted as "persisting" or are removed from the denomina^r in the calculation 
of degree-completion statistics if they withdraw from the institution in good standing or if they 
withdraw andean subsequently be shown to have re-enrolled in another postsecondary institution 
for which the institution from which they have withdrawn provided "substantial prepiu-ation." 
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We make this recommendation for two reasons. First, we believe that removing students 
from an initial starting cohort for any reason will decrease the value of the infurmation produced 
for institutional decisionmaking, and at the same time will yield information of questionable 
value in informing student choice. Second, we wish to caution against combining in the same 
performance statistic information of quite different kinds — institutional persistence, and transfer 
in good standing to another institution — drawn from quite different sources. Persistence at a 
given institution, in the main, can be fairly reported because the institution itself has control over 
the needed data bases. This is not the case for follow-up reporting, which is generally 
accomplished through questionnaire-based methodologies, which often experience difficulties in 
obtaining an adequate response rate. A single statistic based on this combination may as easily 
represent differential response rates among institutions as it will signify real differences in 
institutional performance. 

Instead we recommend that known rates of transfer to other institutions be reported separately 
from institutional persistence and degree-completion rates. The latter should be calculated on a 
denominator base of the cohort as a whole, while the former can be appropriately calculated on 
the same base or alternatively as a percentage of those withdrawing in good standing. Whichever 
alternative is used, response rates should also be reported so that users of the information can 
make an independent assessment of its reliability. 

Proposed Standard Reporting Statistics ^ 

In order to provide both a meaningful picture of student progress nationally and a basis for 
appropriately comparing persistence and degree-completion across institutions, we believe that 
a range of standard performance statistics is required. Some, as noted ai*e tlie kinds of outcomes 
statistics called for by "Student Ri«]ht-to-Know." Others, equally important, provide additional 
information about institutional context which will allow consumers of the ir.'^ormation to 
determine how much of the variance in reported progress is due to the kinds of students served 
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and how much is due to real differences in institutional performance. Recommended standard 

reporting statistics are presented below. 

1. Persistence Indicators . Persistence indicators report the proportion of each s tarting cohort 
remaining enrolled within a given reporting time period. In general, "persistence" is defined 
operationally as attempting at least one credit during the time period reported — term or 
academic year. Within this broad category, we suggest compiling four performance statistics: 

• Percent enrolled at "normal" progress . This statistic represents the proportion of the total 
starting cohort actively enrolled for credit during the reporting period who have 
completed at least 30 student semester credit hours (SCH) or equivalent during the 
academic year. In essence, "normal" progress represents a credit-earning rate that would, 
if continued, result in degree-completion within the time period designated in the college 
catalogue. 

• Percent enrolled, but not at "nornial" progress . This statistic represents the proportion 
of the total starting cohort actively enrolled, but not meeting the definition of "normal" 
progress above. 

• Percent enr olled in good standing . This statistic represents the proportion of the total 
starting cohort actively enrolled for credit during the reporting period who are in good 
academic standing. Normally, this requires maintaining a minimum grade-point of 2.5 
or above, though institutional definitions will be used. 

• Percent enr olled not in good standing . This statistic represents the proportion of the total 
starting cohort enrolled for credit within the reporting period who do not meet the 
definition of "good standing" above. Note that this may include students on probation 
for academic, disciplinary, or fiscal reasons. 

Persistence indicators should be calculated and reported for multiple reporting periods 
within (and beyond) the "150% of normal" time to completion specified by the "Student 
Right-to-Know" legislation. For baccalaureate degree-seekers, reporting periods of four. 
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five, six, and seven or more years are suggested. For associate degree-seekers, reporting 

periods of two through at least six years are appropriate. 
2. Deenree-Completion Indicators . Degree-completion indicators report the proportion of a 
starting cohort successfully completing a program of study within a designated time period. 
In general, completion is defined as the formal award of a degree or certificate. 
Occasionally, however, "completion" has been defined operationally as completing all 
requirements for a given degree, even though it is not formally conferred — an outcome 
frequently encountered in two-year academic transfer programs. Given this condition, we 
suggest three main indicators: 

• Percent completing programs . This statistic represents the cumulative proportion of the 
starting cohort completing programs across all terms prior to the point in time at which 
the statistic is calculated. Consistent with the recommended persistence indicators above, 
we suggest that degree-completion statistics be calculated and reported for multiple 
reporting periods within (and beyond) the "150% of normar' time to completion. Also 
as above, separate reporting of two-year and four-year degrees is appropriate. 

• Average time to degree-completion . This statistic represents the average elapsed time 
required for members of the cohort to complete their programs. In both informing 
"consumers" and institutional planning, it is a useful complement to percent completing 
because it provides a concrete estimate of the expected time commiuTient needed. Unlike 
percent completed within a given time period, moreover, it can reflect data drawn from 
all degree-completers — not merely those who complete within the "150% of normal" 
time-to-completion window. 

• Percent of required credits completed at graduation . This is a contextual statistic 
intended to reflect the "efficiency" with which students complete their programs. 
Increasingly, students are completing more than the number of credits required in their 
programs. Reasons include: (a) the unavailability of required courses and the need to 
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maintain full loads for financial aid or other reasons, and (b) the consequences of 
changing programs and consequently having to take additional required but unplanned 
coursework in a new major field. 
3. Contextual Indicators . Contextual indicators are intended to provide additional 
information about student enrollment behavior that can be used to better interpret the student 
performance statistics described above. They are particularly important in allowing users of 
the information to determine the extent to which obseived outcomes are a function of actual 
institutional performance or of the predominant clientele served by the institution. Two such 
statistics are recommended, each of which reflects a different kind of attendance pattern. 

• Percent attempting "normal" progress . This statistic represents the proportion of the 
starting cohort that enrolls for at least the number of credits each term that, if completed, 
would result in completing the degree sought within the "normal" time expected. It is 
defined operationally by the current standing of the student with respect to 
degree-completion and the credit loads attempted by the student during the terms covered 
by a given reporting period. Its primary utility is to allow users of the performance 
indicators above to assess the extent to which long times to completion are larg-^ly due 
to part-time attendance at the institution in question. 

• Percent at t ending "continuously ." This statistic represents the proportion of the starting 
cohort that has enrolled at the institution for contiguous regular terms of attendance, 
excluding summers, across all terms of a given reporting period. It is defined 
operationally in tenns of the proportion of students who enroll for at least one credit in 
all terms (excluding summers) up to and including the term corresponding to the point 
in time at which the statistic is calculated. Its primary utility is to allow users of the 
performance indicators above to assess the extent to which long times of completion are 
largely due to "stop out" behavior at the institution in question. 



Because of the distinctive missions and clienteles typical of different types of 
postsecondary institutions, quite different rates of persistence and degree-completion can 
be expected regardless of how well a particular in^.titution performs. As a result, we 
additionally recommend that baseline expectations with regard to both persistence and 
degree-completion be established and noted for different types of institutions, based on 
available national datasets that contain consistent information on postsecondary 
persistence and success for a representative cross-section of the population. The data are 
of sufficient reliability to establish average cohort rates of persistence and 
degree-completion by postsecondary sector. Once established, such baseline perfoitnance 
expectations might be used as an average performance standard against which to more 
appropriately compare the performance of any given institution. 

A Proposed Base Dataset 

Most of the above performance statistics are calculated from one or more base data elements 
that must be collected and maintained according to standard national definitions. In many cases, 
such definitions already exist but in many cases they do not. Considerable work will be required 
to develop such definitions as current institutional practices vary widely. 

Our recommendations for a minimal tracking dataset are presented in Chart 2 and are briefly 
discussed below. In choosing data elements to be included, we followed three basic principles: 
(a) that the data element was required for anticipated national reporting such as in "Student 
Right-to-Know;" (b) that the data element is generally included in existing state and institutional 
tracking data bases; and (c) that the data element is fg^nerally defined and maintained in existing 
student records sy.»tems regardless of the type of institution. As with all other proposals in this 
paper, we intend this list not as definitive, but as a starting point for discussion. Most institutions 
can and do maintain far more data elements when they elect to establish a local longitudinal 
tracking capability. National reporting should in no way inhibit this practice. 
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Chart 2 

RECOMMENDED DATA ELEMENTS FOR BASE DATASET 
Initial Descriptors : 

1. Demographic elements 

Sex* 
Race* 

Age at entry* 
Citizenship* 

2. Enrollment status elements 

First temi of academic history (for cohort definition)* 
Admission status* 

- Regular (unrestricted) 

- Provisional/admissions exception 
Degree intended* 

- Associate degree (2-year) 

- Baccalaureate degree (4-year) 
Entering type* 

- First-time freshman 

- Transfer (with <30 SCH; 30-60 SCH; 60-90 SCH; and > 90 SCH transfer credit 
equivalent) 

3. Matriculation status 

4. Athletic status elements 

Recruited as athleJe* 

- By sport 

- By scholarship 

Term Tracking Elements : 

1 . Enrollment/completion 

Current class level (e.g., "freshman")* 
Academic standing* 

- Good standing 

- Probation 

- Suspension (academic) 

- Suspension (other) 

SCH (or equivalent) degree credits enrolled for by lemi* 
SCH (or equivalent) degree credits earned by term* 
Degrees granted* 

- By type (associate vs. baccalaureate) 

- By field of study 

2. Athletic activity 

Participation (by sport)* 
Scholarship received (by sport)* 



*Indicates standard definition currently exists 
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1. Initial descriptors . Initial descriptors should be collected and frozen at the point a 
student enters a cohort. Their primary utility is to allow aggregate institutional performance 
statistics to be broken down by specific populations of interest. Consistent with the 
population reporting requirements of "Student Right-to-Know" and wi h current good 
practice in student tracking, we suggest maintaining initial descriptors of the following 
types: 

Demographics . The four elements proposed are commonly used as student population 
descriptors and all are currently governed by existing state and federal reporting 
definitions. 

Enrollment status . These elements describe key aspects of initial enrollment at the 
institution that may markedly affect future persistence and performance. Proposed 
elements here are included foi different reasons. Admissions status (unrestricted and 
exceptional) is included because large numbers of admissions exceptions in a given 
cohon may markedly influence later performance; we do not advocate excluding such 
students from the cohort but they should be separately reportable. Degree intent and 
entering student type (first-time vs. transfer) are intended as mandatory separate lines for 
reporting all perfonnance statistics as above. Degree intent will both determine cohort 
membership and will provide the basis for assessing "150% of normal" in calculating 
completion. Separate reporting of transfers and first-time freshmen (as well as breaking 
down transfer population reporting by the amount of prior transfer credjt earned in 
academic year equivalents) allows fair comparisons in performance to be made across 
institutions with different incoming ratios between transfers and first- time students. 
Virtually all these elements are already fully or partially defined at the national level. 
Important exceptions are admissions status and proposed distinctions among entering 
transfers. An additional problem, as noted above, is degree declaration. On the one 
hand, students may not declare a degree until substantial numbers of credits have already 
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been accrued, and thus not be included in an initial "degree-seeking" cohort. On the 
other hand, students may "declare" a degree for reasons of administrative convenience 
with no real intention of completing one. 

• Athletic statur .. A major intent of "Student Right-to-Know" is separate reporting of 
student progress for student athletes, broken down by sport. Standard federal definitions 
do not currently exist here, though athletic association guidelines are available as a 
model. 

2. Term tracking data elements . Term tracking data elements provide the raw materials for 
constructing all proposed performance statistics. They are extracted from existing records 
for each term of enrollment across the defined tracking period. Consistent with the 
discussion above, they should be collected in a manner that corresponds to the established 
academic calendar and its associated term structure; necessarily, this means that some 
institutions or state systems will be extracting three terms of data while others may extract 
four or more each academic year. But regardless of academic calendar, term data elements 
should be collected in a manner that allows consistent aggregation into a standard academic 
year. We do not anticipate major difficulty with this concept, as it is already the method 
used in federal IPEDS reporting, for example, degrees completed by field. 
Term tracking data elements recommended for the base data set include: 

• Degree cre dits enrolled for and earned . These are best compiled according to existing 
academic credit-record procedures — quarter hours, semester credit hours, courses, etc. 
— so long as they can be converted to a single standard across an academic year. 
Credits enrolled for are used to determine "persistence" each term and to calculate 
average load or other contextual statistics that might indicate less than full-time 
continuous enrollment. No national standard currently exists for reporting credits. But 
because only persistence will be recorded using this data element as a base, established 
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procedures for determining whether or not a student is '^enrolled" at census (such as uged 
in IPEDS) can be followed here. 

Degree earned . This element indicates whether a degree or certificate is earned during 
a particular term of enrollment and is used to mark "completion" for performance 
reporting. The element should at minimum distinguish among types of degrees that 
require different "normal" times to completion. Ideally, the type and field of the degree 
award should also be recorded. Degree awards should in this case follow established 
federal reporting definitions for level and field of study. 

Academic standing . This element provides contextual information on student 
performancev and is generally reflective of a minimum maintained grade-point-average. 
Proposed categories, would include good standing, probation, academic suspension, and 
suspension for other reasons (e.g., disciplinary). No common federal definitions of this 
^^lement exist currently. Because of wide variations among institutional procedures in 
awarding and recording grades, no common grade-performance reporting is 
recommended. But institutions are strongly encouraged to maintain GPA as a term 
tracking element for their own purposes. 

Current class equivalent level . This element indicates the current class standing of the 
student (e.g., freshman, sophomore, junior, or senior). In principle, this should be 
equivalent to a known number of credits earned, but current procedures differ in this 
respect. Consequently, this element is recommended as an additional means to indicate 
the rate of a given student's progress toward a degree. Because federal definitions 
regarding class level are available, this data element might be used as an alternative to 
developing standard credit equivalents to roughly indicate the proportion of a given 
student's program that has been completed. 

Athletic activity . For some reporting purposes, term records of student athletic 
participation may be needed by sport; in addition, because students may slop receiving 
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athletic scholarship support during their period of enrolhaent, tracking data on 
scholarship support may be appropriate for inclusion. Federal data reporting definitions 
do not currently exist in this area. 

V. Uses of the R es ulting Data 
Information on student persistence and perfonnance has been used for a variety of purposes 
at the state and institutional levels. Indeed, as might be expected, different intended uses have 
led to the construction of quite different longitudinal databases. While it is beyond the scope of 
this paper to review institutional uses of such data, its ready availability based upon standard 
definitions at the state and national levels suggest a number of potential applications. The 
purpose of this section is to briefly review these potential state and federal uses, place some 
important caveats upon them, and to suggest a process for moving forward. 

Some Possible Applications 

While information on comparative institutional perfomance has not been readily available, 

experience suggests several broad uses for information of this kind. They include the following: 
1- To determi ne institutional "eligibility ." Perfonnancc infomiation may first be used to 
help determine if institutions can qualify to receive funds or to otherwise participate in a 
range of state or federal programs. At the federal level, for example, compliance with the 
disclosure provisions of the "Student Right-to-Know" legislation will likely be linked with 
eligibility to receive federal financial aid funds and with eligibility for other Title IV federal 
categorical programs. At the state level, such linkages have already occurred with a number 
of assessment initiatives. In Virginia, for example, institutional compliance with state 
assessment guidelines is linked to eligibility to participate in a range of addition-to-base 
incentive programs. At both levels, it is important to stress, eligibility is tied to procedural 
compliance, not to the actual levels of performance reported and achieved. 
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A different kind of "eligibility" is provided by institutional accreditation. All six regional 
accrediting bodies, for example, now require some demonstration of institutional effectiveness 
through demonstrated outcomes. Currently, this demonstration can be accomplished as the 
institution sees fit — using a range of procedures and measures. The existence of national 
data standards on student progress and completion, however, may result in accrediting 
agencies requesting all institutions to report such data as a standard part of self-study — 
much as basic institutional enrollment profile data is cuiTently requested. 
2. To assist in resource allocation decisionmaking . Information on comparative institutional 
performance can also be used directly to help make resource allocation decisions. In several 
states, for example, such information is used in addition-to-base marginal incentive grant 
programs or to directly reward performance. These cases are, however, of two quite different 
kinds. First, exemplary perfoixnance can be differentially rewarded with the hope that it will 
encourage more of the same. Tennessee's "performance funding" approach is the most 
visible example of this approach, with levels of additional award linked directly to levels of 
indicated performance on a range of outcome measures. In Florida's two-year colleges, 
moreover, institutions must demonstrate minimum levels of job placement in related fields 
in order to continue offering the program. 

A different approach is to use institutional performance information to more broadly 
guide marginal investments cf resources. In New Jersey, for example, one intent of the State 
Unit Record Enrollment (SURE) System and a common collegiate test of "general intellectual 
skills" is to help detennine where available marginal resources should most effectively be 
placed. Under this approach, and in contrast to performance funding, deficiencies in 
performance may result in additional investments to improve performance — particularly if 
the performance area is critical to attaining state goals. A good example here is the goal of 
increasing minority persistence and graduation by directing efforts toward those areas and 
institutions that show lower-than-average performance. 
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3- To provide "consumer" information . Disclosing information on comparative institutional 
performance can potentially also provide an additional foundation for student choice 
regarding which of several available postsecondary institutions to attend. This is the major 
premise, of course, of such initiatives as "Student Right-to-Know," and the several versions 
of "Track Record Disclosure" reporting requirements intended to apply to federally-funded 
occupational/technical programs. The premise of such reporting is that potential students can 
better estimate their personal "return on investment" should they choose to enroll in a given 
program. "Consumer information" requirements of this kind have also been occasionally 
proposed and enacted at the state level. A recent accountability proposal for California's 
community colleges, for instance, is the ability to report to potential stu-'.snts the likelihood 
that they will graduate from a given program and the time that it will probably take them. 
All three of these classes of utilization are, of course, in addition to the more general utility 
of longitudinal data bases for planning, management, and evaluation at the institutional and state 
system level. Most institutions that have developed such systems, for example, do so to help 
improve student retention programs, to project and analyze patterns of enrollment for resource 
planning, or to help evaluate particular academic programs. Parallel uses at the state system level 
include investigating academic progress for particular student constituencies (especially minorities 
or other traditionally underserved populations), assessing the statewide training "pipeline" for 
critical occupations, or evaluating the effectiveness of articulation and transfer arrangements 
among systems of institutions. 

Some Directives Regardint^ Data Use 

The use of performance indicators at any level of decisionmaking in postsecondary education, 
past experience has shown, is a tricky business. This may panicuhu-ly be the case if commonly 
defined performance statistics on student persistence and degree-completion are available and are 
routinely used in "high stakes" decisions. In developing an appropriate approach, therefore, it 
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is essential to concentrate on compiling data that can be used to guide meaningful improvement, 
rather than on creating a statistical reporting system that may subtly redefine what is perceived 
as important. In this regard, both states and institutions should strive toward the development 
of reporting systems that: 

• Encourage managing resources in ways that addi " s real problems rather than toward 
maximizing indicators . This phenomenon, of course, is inherent in all indicator systems 
— particularly those that rely on calculations that are relatively far removed from the 
base data from which they are calculated. Given the complexity of cohort-based 
methodologies, considerable variations in standard performance statistics can be the 
result of relatively small changes in methodology or base data definition. As the recent 
experience of K-12 testing has demonstrated, improving test score performance itself 
can become the paramount management concern if high payoffs are associated directly 
with results. An obvious manifestation in the u e of completion-rate reporting in 
higher education, for instance, may be a proliferation of additional "credentials" for 
attendance periods less than traditional degree requirements. 

• Maintain academic standards . One way to achieve higher rates of graduation is to place 
fewer academic barriers in the path of students on their way to a degree. This is 
exemplary if tiie barriers are unnecessary to real achievement — for example, requiring 
students to take courses or pass examinations that are not related to or are not 
predictive of future peifonnance. But it is a problem if "barriers" represent real 
performance standards which students ought to be able to achieve if a:i awarded degree 
is to retain its integrity. If payoffs for institutions to achieve high rales of 
degree-completion are high, one result may be to relax academic standards of progress. 

• Maintain needed acce.s.s to higher education . An opposite strategy for achieving higher 
rates of graduation is to limit admission to those types of students likely to achieve at 
higher rates. At present, for example, the missions of many public institutions 
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emphasize open access — with the associated understanding that educating the 
open-access student may require greater investments of time and resources than those 
required for liigh ability students. If these differences in mission are not otherwise 
recognized in a system of performance indicators, pressure will be strong for 
"low-performance" institutions to begin examining the kinds of students they admit. 
Keep implementation costs in line . As noted earlier, current institutional capacity to 
report student persistence and degree-completion on a cohort basis is limited. In order 
to develop an adequate reporting capability, most institutions will need to reconfigure 
present student records data bases, and to invest additional staff time to creating, 
updating, and manipulating longitudinal files. While fai* from prohibitive, experience 
has indicated that the costs associated with these activities are also significant. Locd 
benefits of the associated information, we believe, are considerable and certainly a 
growing number of institutions have been willing to inve't in a longitudinal tracking 
capability on its own merits. But policy makers should be aware of the fact that 
considerable investment will be required if most are expected to produce the kinds of 
performance statistics recommended above. 

Provide contextual infom iation that allows users to draw better conclusions about the 
factors responsible for a pparent variations in performance . A major constraint on any 
indicator system is its very simplicity. Considerable research and experience has 
demonstrated that student progress is a complex, multi-causal phenomenon. Similar 
rates of persistence and degree-completion, for example, may be due to quite different 
interactions between student and environmental characteristics. As a result, a "high" 
rate for a two-year college serving a non-traditional student population may be an 
unusually low one for a residential four-year college with selective admission standards. 
Under such circumstances, an indicator system must place a high premium upon 
contextual statistics that allow users to determine appropriar- or expected performance. 
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Even with contextual information, however, the likelihood is considerable that 
differences observed in institutional performances will be misattributed. 

• Encourage the development of best practice at the institutional level . Experience in 
many fields has amply demonstrated the proposition that national standards may be 
radically at odds with local informational utility. Consequently, we have taken special 
care this paper to recommend data definitions and calculation procedures that can 
simultaneously serve national, state, and institutional needs. If multi-purpose reporting 
systems are not developed, we argue, the consequences will be more than simple loss 
of efficiency. If institutions do not use the resulting information on a daily basis, they 
have little incentive to ensure that it is accurate. At the same time, those who do not 
possess a local tracking capability will not be encouraged to develop one beyond the 
minimal limits imposed by federal or state reporting. Both these difficulties represent 
potential dangers, and each should be carefully monitored throughout the process of 
developing appropriate data collection, reporting, and utilization procedures. 

VI. What*s Needed Next? 
The intent of this paper is to raise issues, not propose conclusions. Because we believe that 

the most productive discussions revolve around concrete proposals, we have developed some 

specific recommendations regarding the actual construction and content of minimal longitudinal 

data files and have proposed a minimum set of standard performance statistics that we believe 

will meet the needs of national, state, and institutional constituencies. !n order to further this 

discussion, however, we strongly recommend that proposed reporting statistics and their 

underlying data definitions be formally discussed by appropriate professioiial bodies in four areas: 

• Potential governmental users, such as the National Center for Education Statistics 
(NCES) and state coordinating boards. 

• Professional bodies representing those in institutions of higher education who are 
responsible for data collection, maintenance and reporting; at minimum, this includes 
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the Association foi .stitutional Research (AIR) and the American Association of 
Collegiate Registrars and Admissions Officers (AACRAO). 

• Associations representing the various institutional sectors of postsecondary education, 
to determine the particular problems in performance reporting and data interpretation 
associated with their respective types of institutions; at minimum, this would include, 
the National Association of State Universities and Land Grant Colleges (NASULGC), 
the American Association of State Colleges and Universities (AASCU), the American 
Association of Community and Junior Colleges (AACJC), and the Council of 
Independent Colleges (CIC). 

• Associations representing special interests in reporting student progress, such as the 
National Collegiate Athletic Association (NCAA). 

We also recommend ihat examples of "best practice" in longitudinal student tracking data 
base construction and use be documented as a guide for further development at both the state and 
the institutional levels. At the state level, these should include approaches based on the existence 
of a unit-record data base as well as those based on common procedural guidelines for 
institutional reporting. At the institutional level, examples should be documented from a range 
of institutional types — including public community colleges, public and private four-year 
colleges and universities, and proprietary institutions — of varying sizes and levels of 
organizational complexity. In addition, at the institutional level, approaches should be 
documented that range in technical sophistication and in the required computing and analytical 
capacity. 

Developing consistent approaches to reporting student progress, we believe, is in the interest 
of institutions, states, potential consumers, and the nation as a whole. But no one is served by 
statistics that are misleading, or by procedures that cannot be meaningfully complied with. The 
proposals contained in this paper, we feel, iire workable and appropriate. We urge that they be 
used as a starting point for a national dialogue. 
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